Skip to content

Add comprehensive symmetry testing framework documentation and CI#4

Merged
stevevanhooser merged 7 commits intomainfrom
claude/add-symmetry-tests-U0CE9
Apr 11, 2026
Merged

Add comprehensive symmetry testing framework documentation and CI#4
stevevanhooser merged 7 commits intomainfrom
claude/add-symmetry-tests-U0CE9

Conversation

@stevevanhooser
Copy link
Copy Markdown
Contributor

Summary

This PR significantly expands the cross-language symmetry testing framework for NDR-python ↔ NDR-matlab parity by adding comprehensive documentation, implementation guides, and a new GitHub Actions CI workflow to automate the full test cycle.

Key Changes

  • Enhanced symmetry_tests.md: Completely restructured and expanded the developer documentation with:

    • Clearer architecture diagrams showing the two-phase workflow (makeArtifacts → readArtifacts)
    • Detailed artifact directory layout and naming conventions
    • Step-by-step instructions for writing new symmetry tests in both Python and MATLAB
    • Explanation of why symmetry tests are excluded from default pytest runs
    • Naming conventions table for consistency across languages
  • Expanded ndr_matlab_python_bridge.yaml specification: Reorganized into 6 numbered sections with:

    • Clear guidance on the "Active Maintenance" instruction for agents/developers
    • Detailed procedure for detecting upstream MATLAB changes via matlab_last_sync_hash
    • Complete structure documentation for classes and functions with example YAML blocks
    • Step-by-step procedure for building new bridge YAML files from scratch
    • Summary table of field rules and requirements
  • New GitHub Actions workflow (test-symmetry.yml):

    • Checks out both NDR-python and NDR-matlab repositories
    • Sets up MATLAB, Python, and required dependencies (xvfb, MatBox)
    • Implements the full 4-stage symmetry test cycle:
      1. MATLAB makeArtifacts (generates MATLAB test artifacts)
      2. Python readArtifacts (validates MATLAB artifacts)
      3. Python makeArtifacts (generates Python test artifacts)
      4. MATLAB readArtifacts (validates Python artifacts)
    • Runs on push to main, pull requests, and manual dispatch with concurrency controls
  • New instruction files:

    • tests/symmetry/make_artifacts/INSTRUCTIONS.md: Rules and examples for generating artifacts
    • tests/symmetry/read_artifacts/INSTRUCTIONS.md: Rules and examples for validating artifacts

Notable Implementation Details

  • The workflow uses matlab-actions/setup-matlab@v2 and ehennestad/matbox-actions/install-matbox@v1 for MATLAB environment setup
  • Virtual display server (Xvfb) is required for headless MATLAB execution on Linux
  • Both Python and MATLAB tests use parameterized testing to validate bidirectional compatibility
  • Artifact paths use camelCase for directory components to ensure both languages write to identical paths
  • The matlab_last_sync_hash field enables efficient detection of upstream MATLAB changes for maintenance

https://claude.ai/code/session_01BWkrpUgzWSQx48s8Tk6fnT

Mirror the comprehensive NDI-python bridge spec so NDR developers
have step-by-step guidance for authoring new bridge files, including
the active-maintenance rules, upstream-change detection via
matlab_last_sync_hash, and a section on how to bootstrap a new
bridge YAML file for a new sub-package.
Runs the full 4-stage NDR cross-language symmetry cycle on pushes to
main, pull requests targeting main, and manual dispatch:

  1. MATLAB makeArtifacts       (writes matlabArtifacts/)
  2. Python  readArtifacts       (reads matlabArtifacts/)
  3. Python  makeArtifacts       (writes pythonArtifacts/)
  4. MATLAB readArtifacts       (reads pythonArtifacts/)

This runs separately from the regular ci.yml Python test job so that
it can evolve independently and only executes when the symmetry
suite changes or a full cross-language cycle is explicitly needed.
Both MATLAB stages were failing before any tests ran because:

1. `addpath(genpath("+ndr"))` is invalid: MATLAB namespace (+foo)
   directories must not appear on the path themselves, only their
   parents. Replace with `addpath(ndrRoot)` plus
   `addpath(ndrRoot/tools/tests)` so both the top-level `+ndr` and
   the test `+ndr/+symmetry` packages are discoverable.

2. `ndr_Init` tried to `mkdir` a filecache path under `userpath`
   which isn't writable on the CI runner. The symmetry tests only
   need `ndr.fun.ndrpath()` (self-contained), so skip `ndr_Init`.

Also use an explicit `ndrRoot` variable so the `cd` indirection is
gone and `pwd` stays at the workspace root throughout the job.
Two fixes to get the cross-language symmetry CI actually running:

1. The NDR-matlab checkout was defaulting to `main`, which does not
   contain the `+ndr/+symmetry` package until the feature branch is
   merged. Pass `ref: ${{ github.head_ref || github.ref_name }}` so
   NDR-matlab is checked out at the same branch name as the
   NDR-python change under test (developers are expected to keep
   branch names in sync across the two repos, the same way this
   feature branch does). After both repos merge to main, the same
   expression resolves to `main`.

2. Install NDR-matlab's tools/requirements.txt via
   matbox.installRequirements so vhlab-toolbox-matlab (which
   provides vlt.string.*) is available to the reader.intan_rhd
   symmetry tests in both MATLAB stages.
@stevevanhooser stevevanhooser merged commit 773839f into main Apr 11, 2026
5 checks passed
@stevevanhooser stevevanhooser deleted the claude/add-symmetry-tests-U0CE9 branch April 11, 2026 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant